The evaluation of quality assurance: developing and testing practical methods for managers.

OBJECTIVE OF THE STUDY To develop an approach for evaluating quality assurance (QA) activities and programs in health care settings and to test different evaluation methods. DESIGN This was not a formal scientific study, but rather a research and development (R&D) study, which followed the following steps: (1) reviewing the literature; (2) clarifying critical issues for all key aspects of QA activities; (3) drafting a guide to provide a flexible vehicle for different approaches; (4) testing and adapting the guide as it evolved in three countries; and (5) testing two evaluation tools (self-assessment and appreciative evaluation) in Chile. SETTING AND STUDY PARTICIPANTS The evaluation guide was tested by evaluating QA structures, activities, and programs at the country, regional, and facility levels in Zambia, Niger, and Chile. RESULTS The study resulted in an evaluation guide, which includes an implementation outline, an evaluation matrix, and an appendix of evaluation tools and methods. The guide helps evaluators: agree on a proposed evaluation's scope and design; develop an evaluation methods plan; and address QA history, advocacy, culture, and structure, as well as QA activities and accomplishments. Specific results of the country evaluations in Zambia, Niger, and Chile are presented in separate articles in this supplement. CONCLUSION The QA programs in which the evaluation guide was tested differed in many ways, such as health system structure, decision to focus on particular services, political level implementing QA, policy environment, leadership, and program evolution. The implementation guide presents an outline of the key implementation steps for an evaluation, and includes checklists and model forms (e.g. sample agenda for a team planning meeting, sample list of questions to focus the evaluation). The evaluation matrix presents indicators by QA component and key question, and it enables evaluators to build an approach and select methods. The appendix describes the various tools and methods presented in the first two sections of the evaluation guide.